Network system and network redundancy method

ABSTRACT

In a network system in which a switch forwarding packets and a control server determining route information are separated, it is desired to achieve a redundancy of the control channel by an out-of-band control channel and an in-band control channel, when the switch receives a control message regarding the flow entry registration and the like based on the route information from the control server. Specifically, the separated switch and control server are connected by a control channel for sending and receiving the control message. The switch is not only connected to a control server via an out-of-band control channel by a route dedicated to the control message, but also connected to another control server via an in-band control channel by a route which is common with a normal data communication for determining the route information, to achieve a redundancy of the control channel.

TECHNICAL FIELD

The present invention relates to a network system, and especiallyrelates to a network system where a switch for forwarding a packet and acontrol server for determining route information are separated from eachother.

BACKGROUND ART

In recent years, as one of route control systems in a network system, aroute control method that uses the OpenFlow technique, a controlprotocol for communication devices, has been studied.

In the network system where a route control based on the open flowtechnique is performed, a control server such as an OFC (OpenFlowController) determines the route information, operates a flow table ofthe switch such as an OFS (OpenFlow Switch) in accordance with thedetermined route information, and thereby controls the behavior of theswitch.

The flow table is a table for registering a flow entry that defines apredetermined process (an action) to be executed to a packet conformedto a predetermined matching condition (a rule). A packet group (a packetseries) conformed to the rule is called a flow. The rule of the flow isdefined by various combinations using some or all of: a DestinationAddress; a Source Address; a Destination port; and a Source Port whichare included in a header region of each protocol hierarchy of thepacket, and can be distinguished. Meanwhile, in the above-mentionedaddress, a MAC address (Media Access Control Address) and an IP address(Internet Protocol Address) are included. Further, in addition to theabove description, information on an Ingress Port also can be used asthe rule of the flow.

Details of the OpenFlow technique are described in Non PatentLiteratures 1 and 2.

FIGS. 1 and 2 show a network system including: a switch for forwarding apacket; and a control server for determining route information as anexample of a network system where the route control is performed on thebasis of the OpenFlow technique. FIG. 1 shows a basic configuration ofthe minimum unit of the network system. FIG. 2 shows a specificconfiguration example of the network system.

The network system includes a control server 10 and a switch 20.

The control server 10 can determine route information 11, and canregister a flow entry 22 on a flow table 21 of the switch 20 via acontrol channel.

The switch 20 has the flow table 21, and stores the flow entry 22. Inaddition, the switch 20 forwards a received packet via a link to anotherswitch 20 or to a terminal 30 on the basis of the flow entry 22 storedin the flow table 21.

FIG. 3 shows information registered to the flow entry.

The flow entry 22 retains a match field 51, a priority 52, and an action53 as the information.

The match field 51 is the information used for matching (collation)between the header information of the packet and the flow entry, andhas: the MAC address and the IP address of the source and destinationincluded in the header information; a VLAN ID (Virtual Local AreaNetwork-identifier); and information related to a physical port, anapplication port, and the like. The priority 52 is the informationrelated to a priority order used for determining the matching order ofthe flow entry. The action 53 is the information related to a processmethod (process details) of the packet, and has information defining tosend the packet to a specific port or to discard the packet.

FIG. 4 shows the header information of the packet.

The packet retains as the header information: a MAC header 61; an IPheader 62; a TCP (Transmission Control Protocol) header 63; an encryptedcontrol message 64. As an example of the control message 64, a “FlowMod” message for registering the entry from the controller to the flowtable of the switch, the message being one of the Open Flow ProtocolMessages, and the like can be considered.

The MAC header 61 has the source MAC address (the Src MAC in FIG. 4) andthe destination MAC address (the Dst MAC in FIG. 4). The IP header 62has the source IP address (the Src IP in FIG. 4) and the destination IPaddress (the Dst IP in FIG. 4). The TCP header 63 has the source port(the Src Port in FIG. 4) and the destination port (the Dst Port in FIG.4).

The switch 20 refers to the header information extracted from the packetheader of the received packet and to the match field 51 of the flowentry, and when the information on the same items included in themconform by the matching respectively, the switch 20 determines theaction 53 of the flow entry as the action to the packet.

In FIG. 5, regarding the flow entry in each of the flow tables, thematching order is determined on the basis of the priority of each flowentry.

As described above, in the above-described network system, the switchused as the packet forwarding means and the control server used as theroute information determination means are separated from each other, aconfiguration where one control server determines the route informationof the plurality of switches is realized, and the switch and the controlserver are connected by a control channel. In this case, in order toprevent the communication between the switch and the control server fromdelaying and being interrupted, it is desired that the control channelis a dedicated line; however, a dedicated port for the switch and adedicated link for the network are required to use the control channelas the dedicated line, and thus availability and expandability arerestricted. In addition, in the case where the dedicated line of thecontrol channel is disconnected, the switch fails to receive the controlinformation.

As a related technique, Patent Literature 1 (JP2000-078194A) discloses anetwork system. In the related technique, a switch composing the networkincludes a plurality of connection ports, and to the ports, atransmission path, an end system, and a network server each composingthe network are connected.

In addition, Patent Literature 2 (JP2003-273907A) discloses anautonomous system, a communication control method, a server, and arouter. In the related technique, the autonomous system is configured byincluding: a plurality of BGP routers having a function for performingcommunication based on a BGP (Boarder Gateway Protocol) mutually withanother autonomous system; a single server for intensively controllingthe BGP communication due to the BGP router; and a router for relayingthe communication between the server and the BGP router without havingthe communication function based on the BGP. The BGP router and therouter can change the route information of an IP packet in itself inaccordance with an order from the server.

Moreover, Patent literature 3 (JP2007-251344A) discloses a wirelesscommunication device and a wireless communication method. In the relatedtechnique, even in the case where upstream control information includedin the control information cannot be obtained because of occurrence ofan error in a control channel, the upstream control information isplaced in-band in a data channel, and accordingly a frame for theupstream transmission is created by using the upstream controlinformation placed in-band.

CITATION LIST Patent Literature

-   [PTL 1] Patent literature 1: Japanese Patent Application Publication    JP2000-078194A-   [PTL 2] Patent literature 2: Japanese Patent Application Publication    JP2003-273907A-   [PTL 3] Patent literature 1: Japanese Patent Application Publication    JP2007-251344A

Non-Patent Literature

-   [NPTL 1] Non patent literature 1: “The OpenFlow Switch    Consortium”<http://www.openflowswitch.org/>-   [NPTL 2] Non patent literature 2: “OpenFlow Switch Specification    Version 1.0.0 (Wire Protocol 0x01) Dec. 31,    2009”<http://www.openflowswitch.org/documents/openflow-spec-v1.0.0.pdf>

SUMMARY OF INVENTION

A mechanism is provided, by which a switch has an ability to connect notonly the out-of-band control server but also other control servers suchas the in-band control server, and the control channel acquiresredundancy by the out-of-band control server and the in-band controlserver.

A network system according to the present invention includes: a switch;an out-of-band control server; and an in-band control server. The switchforwards a packet. The out-of-band control server sends a controlmessage to the switch via an out-of-band control channel to determineroute information. The in-band control server sends a control message tothe switch via an in-band control channel to determine routeinformation. Further, the out-of-band control server registers a flowentry for control message by which the switch is defined to forward acontrol message to the in-band control server. Here, each of the switch,the out-of-band control server, and the in-band control server is acomputer.

In a network redundancy method according to the present invention, apacket is forwarded by a switch. A control message is sent from anout-of-band control server to the switch via an out-of-band controlchannel to determine route information. A control message is sent froman in-band control server to the switch via an in-band control channelto determine route information. Further, a flow entry for controlmessage by which the switch is defined to forward a control message isregistered from the out-of-band control server to the in-band controlserver.

A program according to the present invention is a program for making acomputer which functions as each of the switch, the out-of-band controlserver, and the in-band control server execute the above-mentionednetwork redundancy method. The program according to the presentinvention is able to be stored in a storage device or a storage medium.

In a network system in which a switch for forwarding packets and acontrol server which determines route information are separated, theredundancy by the in-band control channel is achieved.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual diagram showing a basic configuration of theminimum unit of a network system;

FIG. 2 is a conceptual diagram showing a specific configuration exampleof the network system;

FIG. 3 is a diagram showing an example of information registered to aflow entry;

FIG. 4 is a diagram showing an example of header information of apacket;

FIG. 5 is a diagram showing an example of a matching order based onpriority of the flow entry;

FIG. 6A is a block diagram showing a basic configuration of the networksystem of the present invention;

FIG. 6B is a block diagram showing a basic configuration of the networksystem of the present invention;

FIG. 7 is a diagram showing an example of a flow entry for controlmessage, the flow entry being registered to a flow table of the switch;

FIG. 8 is a block diagram showing a detailed configuration of an insideof the switch;

FIG. 9 is a diagram showing an example of switch information registeredto a switch information management unit;

FIG. 10 is a diagram showing an example of control server informationregistered to a control server information management unit;

FIG. 11 is a flowchart showing an operation on a side of an out-of-bandcontrol server;

FIG. 12 is a flowchart showing an operation on a side of an in-bandcontrol server;

FIG. 13 is a flowchart showing an operation in establishment of anout-of-band control channel;

FIG. 14 is a flowchart showing an operation in establishment of anin-band control channel;

FIG. 15 is a flowchart showing an operation in registration of theswitch information to the in-band control server;

FIG. 16A is a flowchart showing an operation in switching between theout-of-band control server and the in-band control server;

FIG. 16B is a flowchart showing the operation in switching between theout-of-band control server and the in-band control server;

FIG. 17 is a flowchart showing an operation in matching of the flowentry for control message; and

FIG. 18 is a flow chart showing an operation in extraction of thecontrol message from a packet group.

BEST MODE FOR CARRYING OUT THE INVENTION First Embodiment

Referring to attached drawings, a first embodiment of the presentinvention will be explained below.

[Basic Configuration]

As shown in FIG. 6A and FIG. 6B, a network system of the presentinvention includes an out-of-band control server 100, a switch 200, andan in-band control server 300.

The out-of-band control server 100 has a switch information managementunit 110, a redundancy control server management unit 120, a controlchannel connection unit 130, a control message processing unit 140, androute information 150.

The switch 200 has a control server information management unit 210, acontrol channel connection unit 230, a control message processing unit240, and a flow table 260.

The in-band control server 300 has a switch information management unit310, a redundancy control server management unit 320, a control channelconnection unit 330, a control message processing unit 340, and routeinformation 350.

The out-of-band control server 100, the switch 200, and the in-bandcontrol server 300 have functions to control each device inside andoutside a computer, to generate and process data, and to execute aprogram. For example, the case where: the out-of-band control server 100is outside a route for forwarding a packet; and the switch 200 and thein-band control server 300 are on the route for forwarding the packetcan be considered. On this occasion, the out-of-band control server 100may send only a control message to the switch in the out-of-band method,and the in-band control server 300 may send the control message and datato the switch in the in-band method.

The switch information management unit 110, the redundancy controlserver management unit 120, the control server information managementunit 210, the switch information management unit 310, and the redundancycontrol server management unit 320 have a function for retainingpackets.

The redundancy control server management unit 120, the control channelconnection unit 130, the control channel connection unit 230, theredundancy control server management unit 320, and the control channelconnection unit 330 have a function for sending and receiving the packetto and from the outside via the network.

The control message processing unit 140, the control message processingunit 240, and the control message processing unit 340 have functions forgenerating and processing the control message.

[Exemplification of Hardware]

As examples of the out-of-band control server 100 and the in-bandcontrol server 300, a computer such as a PC (Personal Computer), anappliance, a work station, a main frame, and a super computer areassumed. It is preferred that the out-of-band control server 100 and thein-band control server 300 basically have a same configuration.

As an example of the switch 200, an OpenFlow switch is assumed. As anexample of apparatuses usable for the OpenFlow switch, a relay apparatussuch as a router and a switching hub, a relaying computer such as agateway, a proxy, a firewall, and a load balancer, and the like can beconsidered. For example, the switch 200 may be a multi-layer switch. Themulti-layer switch is further minutely classified every layer of asupported OSI reference model. Major classifications are a layer3 switchfor reading data of a network layer (the 3rd layer), a layer4 switch forreading data of a transport layer (the 4th layer), and a layer7 switch(an application switch) for reading data of an application layer (the7th layer).

The out-of-band control server 100, the switch 200, and the in-bandcontrol server 300 may be a Virtual Machine (VM) structured on aphysical machine.

As examples of hardware for realizing the data process function and theprogram execution function of: the out-of-band control server 100, theswitch 200, the in-band control server 300, the control messageprocessing unit 140, the control message processing unit 240, and thecontrol message processing unit 340, a CPU (Central Processing Unit), amicroprocessor, a microcontroller, a semiconductor integrated circuit(Integrated Circuit (IC)) having the equivalent function, and the likecan be considered.

As examples of hardware for realizing: the data retention function ofthe switch information management unit 110, the redundancy controlserver management unit 120, the control server information managementunit 210, the switch information management unit 310, and the redundancycontrol server management unit 320, and the data retention function forretaining the route information 150, the flow table 260, and the routeinformation 350, a semiconductor storage device such as a RAM (RandomAccess Memory), a ROM (Read Only Memory), an EEPROM (ElectricallyErasable and Programmable Read Only Memory), and a flash memory, anauxiliary storage device such as an HDD (Hard Disk Drive) and an SSD(Solid State Drive), or a removable disk and a storage medium such as aDVD (Digital Versatile Disk) and an SD memory card (Secure Digitalmemory card) can be considered. Meanwhile, the above-mentioned hardwarefor realizing the data retention function is not limited to a storagedevice incorporated in the computer body, and may be a peripheralapparatus (an external HDD and the like), a storage device installed inan external server (a Web server, a file server, and the like), a DAS(Direct Attached Storage), a FC-SAN (Fiber Channel-Storage AreaNetwork), a NAS (Network Attached Storage), or an IP-SAN (IP-StorageArea Network).

As examples of hardware for realizing the communication function of: theredundancy control server management unit 120; the redundancy controlserver management unit 320; the control channel connection unit 130; thecontrol channel connection unit 230; and the control channel connectionunit 330, a network adapter such as a NIC (Network Interface Card), acommunication device such as an antenna, a communication port such as aconnection port (a connector), and the like can be considered. Inaddition, as examples of the network, the Internet, a LAN (Local AreaNetwork), a wireless LAN, a WAN (Wide Area Network), the backbone, acable television (CATV) line, a landline phone network, a mobile phonenetwork, the WiMAX (IEEE 802.16a), the 3G (3rd Generation), a leaseline, the IrDA (Infrared Data Association), the Bluetooth (a registeredtrademark), a serial communication line, a data bus, and the like can beconsidered.

However, the embodiments are not limited the above-mentioned examplespractically.

[Details of Configuration]

The out-of-band control server 100 registers, as out-of-band subordinateswitch information 111, information on the switch (a switch ID and thelike) that is preliminarily managed by itself to the switch informationmanagement unit 110. In addition, the out-of-band control server 100registers information on other control servers existing on the network.With use of the redundancy control server management unit 120, theout-of-band control server 100 inquires other control server whether ornot to be able to provide the service as the in-band control server 300.In the case where other control server notified that the serviceprovision is available, the out-of-band control server 100 uses acertification received from the other side (other control server), andestablishes an encrypted communication route between the out-of-bandcontrol server 100 and the in-band control server 300 with employment ofthe other control server as the in-band control server 300. Moreover,with use of the control message processing unit 140, the out-of-bandcontrol server 100 generates a control message related to the flow entryregistration and the like on the basis of the route information 150, andsends the control message to the switch 200 via the control channelconnection unit 130.

Since receiving the control message mainly from information 211 on theout-of-band control server, the switch 200 registers at least theinformation 211 on the out-of-band control server 211 to the controlserver information management unit 210. With use of the control channelconnection unit 230, the switch 200 sends a connection request to thealready-known out-of-band control server 100. The switch 200 creates acommon key (a shared key) between the switch 200 and the out-of-bandcontrol server 100 on the basis of the certification received from theout-of-band control server 100, and establishes the encryptedcommunication route and the out-of-band control channel. Here, thenetwork for data forwarding of the switch 200 used as a link is adifferent network from the network between the switch 200 used as theout-of-band control channel and the out-of-band control server 100. Theswitch 200 uses the out-of-band control channel as a communication routededicated to the control message.

When the out-of-band control channel has been established between theswitch 200 and the out-of-band control server 100, information 212 onthe in-band control server 300 is notified from the out-of-band controlserver 100 to the subordinate switch 200, and the flow entry thatdefines the packet forwarding to the in-band control server 300 isregistered to the flow table 260 of the switch 200. The flow entryserves as a flow entry 261 for control message, and is set so as to bethe highest priority and to avoid the aging out from the flow tableinside the switch 200. That is, to the flow table 260 of the subordinateswitch 200, the out-of-band control server 100 registers the flow entryshowing that a packet is forwarded to the in-band control server 300 toinquire the route information. In the case where the in-band controlserver 300 is on the packet forwarding route, the in-band control server300 serves as a destination of the packet and also as the controlserver, for the switch 200.

The switch 200 extracts a packet of the control message mixed with othercommunication data by using: the header information of the receivedpacket; and the flow entry for control message, and thereby judgingwhether it is the communication addressed to itself or the relay of thecontrol message. In addition, the switch 200 uses the header informationof the received packet and the flow entry for control message also indetermination of a route for communication between the switch 200 andthe in-band control server 300.

FIG. 7 shows an example of the flow entry 261 for control messageregistered in the flow table 260 of the switch 200. Here, as the exampleof the flow entry 261 for control message, flow entries 401 to 404 areshown. The flow entry 401 is a flow entry for connection to the in-bandcontrol server 300. The flow entry 402 is a flow entry addressed fromthe in-band control server 300 to the switch 200. The flow entry 403 isa flow entry for relay from the in-band control server 300 to the switch200. The flow entry 404 is a flow entry for the in-band control server300.

With use of the out-of-band control channel, the out-of-band controlserver 100 registers the flow entries 401 to 404 for control message tothe flow table 260 of the subordinate switch 200.

The out-of-band control server 100 registers, to the flow table 260 ofthe subordinate switch 200, data that records: the source IP address(Src IP) of the in-band control server; and the TOP port for controlmessage (Src Port) in a match field of the flow entry 401 for connectionto the in-band control server 300. The switch 200 specifies a packet tothe in-band control server 300 by preferentially matching informationextracted from the packet header with the match field of the flow entry,and sends the packet to the in-band control server 300. On the contrary,in the case of specifying a packet from the in-band control server 300,since the Source IP address (Src IP) of the in-band control server 300and the TCP port (Src Port) for control message of the source arerecorded in the match fields of the flow entries 402 and 403, the switch200 can judge whether it is the communication addressed to itself or therelay of the control message on the basis of the packet of the mixedcommunication data.

FIG. 8 shows details of an internal configuration of the switch 200.Here, configurations of: the internal flow table 260 of the switch 200,the flow entry 261 for control message, and the flow entry 262 for dataplane are especially shown. Meanwhile, a flow forwarding unit 270 newlyshown in FIG. 8 receives a data traffic where a packet of the controlmessage and other packets are mixed, and forwards each of the packets onthe basis of the flow entry in the flow table 260. In addition, theinterface 280 has: a dedicated port for the out-of-band control channel,and a universal port for the in-band control channel.

In FIG. 8, the control message via the out-of-band control channel isdirectly sent to the control channel connection unit 230 via a dedicatedport of the interface 280 without matching with the flow entry in theflow table 260. However, the control message via the in-band controlchannel is mixed with other packets and reaches the flow forwarding unit270 of the switch 200 via a universal port of the interface 280. On thisoccasion, with use of the high priority flow entry 261 for controlmessage, the switch 200 specifies the control message, and judgeswhether to forward the control message to the control channel connectionunit 230 addressed to itself or to relay the control message.

With use of: information (IP address and the like) on the in-bandcontrol server 300 existing in the control server information managementunit 210; and the control message processing unit 240, the switch 200requests the in-band control server 300 to connect with via the controlchannel connection unit 230. The in-band control server 300 certifiesthe switch 200, and sends the certification to the permitted switch 200.The switch 200 creates a common key between the switch 200 and thein-band control server 300 from the certification received from thein-band control server 300, and establishes the encrypted communicationroute and the in-band control channel. The in-band control server 300stores the information on the switch in the switch informationmanagement unit 310 as in-band subordinate switch information 312.

The switch 200 monitors statuses of the out-of-band control server 100and of the in-band control server 300, simultaneously receiving thecontrol message related to the flow entry registration and the likebased on the route information mainly from the out-of-band controlserver 100. In the case where abnormality of the status of theout-of-band control server 100 is detected, the switch 200 can instantlyswitch the server to the in-band control server 300.

FIG. 9 shows an example of the switch information registered to theswitch information management unit 110 and the switch informationmanagement unit 310. Here, the switch information registered to theswitch information management unit 110 will be explained.

The switch information management unit 110 has the out-of-bandsubordinate switch information 111 and in-band subordinate switchinformation 112. The out-of-band subordinate switch information 111includes switch IDs 1111 and 1112 and a certification/public key 1113.The in-band subordinate switch information 112 includes switch IDs 1121and 1122 and a common key between server-server 1123.

FIG. 10 shows an example of the control server information registered tothe control server information management unit 210.

The control server information management unit 210 has: the information211 on the out-of-band control server, and the information 212 on thein-band control server. The information 211 on the out-of-band controlserver includes: a MAC address 2111 of the control server, an IP address2112 of the control server, a TCP port number 2113 of the controlserver, and a common key between server-switch 2114. The information 212on the in-band control server includes: a MAC address 2121 of thecontrol server, an IP address 2122 of the control server, a TCP portnumber 2123 of the control server, and a common key betweenserver-switch 2124.

Then, using time charts shown in FIG. 11 to FIG. 18, operations of theserver and so on in the network system of the present invention will beexplained.

[Operation on Out-of-Band Control Server Side]

FIG. 11 shows an operation on a side of the out-of-band control server100.

(1) Step S101

The out-of-band control server 100 selects the in-band control server300.

(2) Step S102

The out-of-band control server 100 requests to connect to the selectedin-band control server 300. Here, the out-of-band control server 100requests the selected in-band control server 300 to provide the serviceas the redundancy control server.

(3) Step S103

The out-of-band control server 100 confirms whether or not a reply ofpermission is issued from the in-band control server 300. Here, in thecase of receiving the certification from in-band control server 300, theout-of-band control server 100 judges that the reply of permission isissued from the in-band control server 300.

(4) Step S104

In the case where the reply of permission is not issued from the in-bandcontrol server 300 (No at step S103), the out-of-band control server 100selects another in-band control server 300, and requests again toconnect to the selected in-band control server 300.

(5) Step S105

In the case where the reply of permission is issued from the in-bandcontrol server 300 (Yes at step S103), the out-of-band control server100 creates a common key with use of the certification from the in-bandcontrol server 300, and establishes the encrypted communication routebetween the out-of-band control server 100 and the in-band controlserver 300.

(6) Step S106

The out-of-band control server 100 notifies: information on thesubordinate switch (a switch ID); and the created common key of thein-band control server 300.

(7) Step S107

The out-of-band control server 100 notifies information on the in-bandcontrol server 300 of the subordinate switch 200.

(8) Step S108

The out-of-band control server 100 registers the flow entry 261 forcontrol message to the flow table 260 of the switch 200.

[Operation on in-Band Control Server Side]

FIG. 12 shows an operation on a side of the in-band control server 300.

(1) Step S201

The in-band control server 300 receives a connection request as thein-band control server 300 from other control servers. Here, therequestor control server is the out-of-band control server 100. Thein-band control server 300 receives a service provision request as theredundancy control server from the out-of-band control server 100.

(2) Step S202

The in-band control server 300 judges whether or not to permit therequestor control server to connect as the in-band control server 300.

(3) Step S203

In the case of not permitting the requestor control server to connect asthe in-band control server 300, the in-band control server 300 notifiesthe requestor control server of the connection refusal.

(4) Step S204

In the case of permitting the requestor control server to connect as thein-band control server 300, the in-band control server 300 notifies therequestor control server of the connection permission, and sends thecertification.

(5) Step S205

The in-band control server 300 receives the common key from therequestor control server, and establishes an encrypted communicationroute and a server-to-server communication route. Here, the in-bandcontrol server 300 receives the common key created by the out-of-bandcontrol server 100 from the out-of-band control server 100, andestablishes the encrypted communication route between the out-of-bandcontrol server 100 and the in-band control server 300.

(6) Step S206

In addition, the in-band control server 300 stores information on theswitch received from the other-side control server. Here, the in-bandcontrol server 300 receives the switch ID showing the subordinate switch200 from the out-of-band controls server 100, and manages the switch ID.Meanwhile, the in-band control server 300 may receive the common key andthe information on switch (the switch ID) at the same timing.

[Establishment of Out-of-Band Control Channel]

FIG. 13 shows an operation in: confirming the TCP session with respectto the out-of-band control server 100 whose switch 200 is already known;creating the public key with use of the certification received from theserver; and establishing an encrypted communication route and anout-of-band control channel.

(1) Step S301

The switch 200 establish the TCP session with respect to thealready-known out-of-band control server 100.

(2) Step S302

The switch 200 receives the certification from the out-of-band controlserver 100, and creates the common key between the out-of-band controlserver 100 and the switch 200.

(3) Step S303

The switch 200 establishes the encrypted communication route between theout-of-band control server 100 and the switch 200.

(4) Step S304

The switch 200 sends a connection request to the out-of-band controlserver 100.

(5) Step S305

The switch 200 confirms whether or not the out-of-band control channelcan be established. Here, when a reply of connection permission isissued from the out-of-band control server 100, the switch 200 judgesthat the out-of-band control channel can be established. In the casewhere the out-of-band control channel cannot be established, the switch200 repeats the operation from the beginning.

(6) Step S306

In the case where the out-of-band control channel can be established,the switch 200 establishes the out-of-band control channel between theout-of-band control server 100 and the switch 200.

[Establishment of in-Band Control Channel]

FIG. 14 shows an operation in establishing the in-band control channelafter the switch 200 established the out-of-band control channel.

(1) Step S401

The switch 200 establishes the out-of-band control channel between theswitch 200 and the out-of-band control server. This procedure is shownin FIG. 13.

(2) Step S402

After the establishment of the out-of-band control channel, the switch200 requests the information on the in-band control server 300 and thecommon key between the out-of-band control server 100 and the in-bandcontrol server 300 to the out-of-band control server 100.

(3) Step S403

The switch 200 receives information on the proposed in-band controlserver 300 from the out-of-band control server 100.

(4) Step S404

The switch 200 sends the own switch ID and the common key between theout-of-band control server 100 and the in-band control server 300 to theproposed in-band control server 300, and requests the in-band controlserver 300 to connect with.

(5) Step S405

The switch 200 confirms whether or not the connection to the in-bandcontrol server 300 is permitted. Here, in the case of receiving a replyof connection permission from the in-band control server 300, the switch200 judges that the connection to the in-band control server 300 ispermitted.

(6) Step S406

In the case of not receiving the reply of connection permission from thein-band control server 300 (No at Step S405), the switch 200 notifiesthe out-of-band control server 100 that “impossible to establish thein-band control channel”, and requests the information on anotherin-band control server 300 again.

(7) Step S407

In the case of receiving the reply of connection permission from thein-band control server 300 (Yes at Step S405), the switch 200establishes the encrypted communication route between the switch 200 andthe in-band control server 300. Here, the switch 200 receives thecertification as the reply of connection permission from the in-bandcontrol server 300, creates the common key between the switch 200 andthe in-band control server 300, and establishes the encryptedcommunication route between the switch 200 and the in-band controlserver 300 with use of the created common key.

(8) Step S408

The switch 200 establishes the in-band control channel between theswitch 200 and the in-band control server 300.

[Registration of Switch Information to in-Band Control Server]

FIG. 15 shows an operation in registering the information on the switch200 of the in-band control server 300.

(1) Step S501

The in-band control server 300 receives a request of establishment ofthe in-band control channel from the subordinate switch 200.

(2) Step S502

In receiving the request from the switch 200, the in-band control server300 confirms the switch information management unit 310 whether or notthe information on the switch 200 is registered. Then, since theinformation on the switch 200 (the switch ID) was previously exchangedbetween the out-of-band control server 100 and the in-band controlserver 300, it is judged whether to accept the registration of theinformation on the switch 200 (the switch ID) or not, referring to theinformation on the switch 200 (the switch ID) and the common key betweenthe out-of-band control server 100 and the in-band control server 300.

(3) Step S503

In the case where the information on the switch 200 (the switch ID) isnot registered (No at Step S502), the in-band control server 300notifies the switch 200 of the connection refusal.

(4) Step S504

In the case where the information on the switch 200 (the switch ID) isregistered (Yes at Step S502), the in-band control server 300 confirmswhether or not the common key between the out-of-band control server 100and the in-band control server 300 is correct.

(5) Step S505

In the case where the common key between the out-of-band control server100 and the in-band control server 300 is not correct (No at step S504),the in-band control server 300 notifies the switch 200 of the connectionrefusal.

(6) Step S506

In the case where the common key between the out-of-band control server100 and the in-band control server 300 is correct (Yes at step S504),the in-band control server 300 notifies the switch 200 of the connectionpermission, and sends the certification to the switch 200.

(7) Step S507

As a reply to the sending of the certification, the in-band controlserver 300 receives the common key between the switch 200 and thein-band control server 300 from the switch 200, and establishes theencrypted communication route with use of the common key between theswitch 200 and the in-band control server 300.

(8) Step S508

The in-band control server 300 establishes the in-band control channelbetween the switch 200 and the in-band control server 300.

[Switch Between Out-of-Band Control Server and in-Band Control Server]

FIGS. 16A and 16B show an operation where the switch 200 switches fromthe out-of-band control serve 100 to the in-band control server 300, orswitches from the in-band control server 300 to the out-of-band controlserve 100.

(1) Step S601

After connecting to each of the out-of-band control server 100 and thein-band control server 300, the switch 200 receives the control messagemainly from the out-of-band control server 100.

(2) Step S602

The switch 200 periodically sends a status monitoring message to theout-of-band control server 100.

(3) Step S603

The switch 200 confirms whether or not a reply to the status monitoringmessage is issued from the out-of-band control server 100. In the casewhere the reply is issued from the out-of-band control server 100 (Yesat step S603), the switch 200 receives the control message related tothe status monitor from the out-of-band control server 100.

(4) Step S604

In the case where the reply is not issued from the out-of-band controlserver 100 (No at step S603), the switch 200 determines that theout-of-band control server 100 is in an abnormal status, and selects thein-band control server 300 to prepare the switching to the selectedin-band control server 300.

(5) Step S605

The switch 200 sends a switch notification to the in-band control server300, and requests the connection initialization to connect to thein-band control server 300.

(6) Step S606

The switch 200 accordingly receives the control message from the in-bandcontrol server 300 at the timing when the connection to the in-bandcontrol server 300 has completed. The switch 200 receives the controlmessage from the in-band control server 300.

(7) Step S607

Meanwhile, even the switch 200 under the connection to the in-bandcontrol server 300 periodically sends the status monitoring mess age tothe out-of-band control server 100.

(8) Step S608

The switch 200 confirms to the status monitoring message whether a replyis issued or not from the out-of-band control server 100. The switch 200checks the status of the out-of-band control server 100 on the basis ofexistence and nonexistence of the reply, and confirms whether theout-of-band control server 100 is restored or not.

(9) Step S609

In the case where the switch 200 has been confirmed: the issue of thereply from the out-of-band control server 100; and the restoration (Yesat step S608), the switch 200 instantly switches to return to theout-of-band control server 100, and receives the control message fromthe out-of-band control server 100.

(10) Step S610

In the case where the reply is not issued from the out-of-band controlserver 100 and the restoration has not been determined (No at stepS608), the switch 200 sends the status monitoring message to the in-bandcontrol server 300.

(11) Step S611

The switch 200 confirms to the status monitoring message whether a replyis issued or not from the in-band control server 300. In the case wherethe reply is issued from the in-band control server 300 (Yes at stepS611) the switch 200 receives the control message related to the statusmonitor from the in-band control server 300. Additionally, in the casewhere the reply is not issued from the in-band control server 300 (No atstep S611), the switch 200 judges that the in-band control server 300 isin an abnormal status, and selects another in-band control server 300 toprepare the switching to the selected in-band control server 300.

[Matching of Flow Entry for Control Message]

FIG. 17 shows a mechanism for matching the flow entry for controlmessage registered on the switch 200, the flow entry being addressed tothe in-band control server 300.

(1) Step S701

The out-of-band control server 110 sends the information on the in-bandcontrol server 300 to the switch 200, and registers the flow entry tothe in-band control server 300. The registered flow entry serves as theflow entry for control message. The switch 200 generates the controlmessage to the in-band control server 300, and sends a packet of thecontrol message.

(2) Step S702

The switch 200 performs the matching with the high priority flow entryfor control message with use of a search key extracted from a header ofthe packet of the control message.

(3) Step S703

The switch 200 confirms whether or not the search key extracted from theheader of the packet of the control message matches with the highpriority flow entry for control message.

(4) Step S704

In the case where the matching was success, the switch 200 sends thecontrol message to the in-band control server 300.

(5) Step S705

In the case where the matching is not success (No at step S703), theswitch 200 confirms whether or not the search key extracted from theheader of the packet of the control message matches another flow entryfor control message. Here, with use of the search key extracted from theheader of the packet of the control message, the switch 200 performs thematching with the high priority flow entry for control message next tothe flow entry for control message to which the matching is performed infirst. In the case where the matching is success, the switch 200 sendsthe control message to the control server corresponding to the flowentry for the control message. In the case where there is no flow entryfor control message matching the search key extracted from the header ofthe packet of the control message, the switch 200 may discard the packetthe control message.

[Extraction of Control Message from Packet Group]

FIG. 18 shows a mechanism for matching the flow entry for controlmessage, the flow entry being for extraction of the control messagemixed with the packet on the flow table on the switch 200.

(1) Step S801

The switch 200 receives the packet.

(2) Step S802

The switch 200 performs the matching with the high priority flow entryfor control message with use of the search key extracted from thereceived packet.

(3) Step S803

The switch 200 confirms whether or not the search key extracted from theheader of the received packet matches the high priority flow entry forcontrol message.

(4) Step S804

In the case where the matching is not success (No at step S803), theswitch 200 confirms whether or not the search key extracted from theheader of the received packet matches another flow entry for controlmessage. Here, with use of the search key extracted from the header ofthe received packet, the switch 200 performs the matching with the highpriority flow entry for control message next to the flow entry forcontrol message to which the matching is performed in first. In the casewhere the matching is success, the switch 200 sends the control messageto the control server corresponding to the flow entry for the controlmessage. In the case where there is no flow entry for control messagematching the search key extracted from the header of the receivedpacket, the switch 200 may discard the received packet.

(5) Step S805

In the case where the matching is success (Yes at S803), the switch 200determines the packet is the control message, and confirms whether ornot the packet is addressed to itself.

(6) Step S806

In the case where the packet is not addressed to itself (No at stepS805), the switch 200 relays the control message to another switch onthe basis of the address.

(7) Step S807

In the case where the packet is addressed to itself (Yes at step S805),the switch 200 forwards the control message addressed to itself to owncontrol message processing unit 240.

Second Embodiment

A second embodiment of the present invention will be explained below.

In the case where a destination port number or a source port of TCP isdefined in a protocol of a control message process (for example, TCPport number is 9999), the switch 200 can extract a control message froma flowing data traffic by using a flow entry for control messagedetection.

The control message is sent to own out-of-band control server, theout-of-band control server can extract a source IP address of othercontrol servers from a header of the control message, and a connectionto other control server can be tried.

[Features of the Present Invention]

The present invention realizes redundancy of the control channel due tothe out-of-band control server and to the in-band control server, in theswitch for forwarding a packet and the control server for determining aroute.

The present invention has a mechanism for registering a flow entry forcontrol message, from the out-of-band control server to the in-bandcontrol server, to the switch.

In the present invention, the switch ID is exchanged between theout-of-band control server and the in-band control server.

Additionally, in the present invention, the in-band control serverregisters another switch.

Moreover, in the present invention, a control message mixed with thepacket is extracted with use of the flow entry for control message.

The present invention can be used in a network system and the like wherepacket forwarding means (the switch) and route information determinationmeans (the control server) are separated from each other.

In the present invention, other control servers can be used as thein-band control server via a network of a data traffic with the switchreceiving the control message from the main out-of-band control server.

Additionally, in the present invention, by registering the informationon the in-band control server to the switch, the in-band control serverbeing preliminarily requested to be connected with from the out-of-bandcontrol server, the switch can connect to the reliable other controlservers (the in-band control server).

Moreover, in the present invention, the in-band control serverpreliminarily receives and stores: the switch ID; and the common keybetween server-server from another out-of-band control server, andthereby when the connection is requested by the switch, certifying theswitch.

Furthermore, in the present invention, the high priority flow entry forcontrol message is registered to the flow table of the switch from theout-of-band control server, and thereby the switch can extract a controlmessage mixed with the data traffic.

Additionally, in the present invention, the control message ispreferentially matched with due to the flow entry for control message,can be sent to the control message processing unit of the switch, andadditionally the control message can be preferentially relayed toanother switch.

Then, in the present invention, in the case where a trouble occurred ina monitored out-of-band control server, the switch switches from theout-of-band control server to the in-band control server once, andreceives the control message from the in-band control server; however,by monitoring the out-of-band control server before the switching, whenthe out-of-band control server is restored, the switching can beinstantly switched to return.

<Summary>

As described above, in the network system where the switch forforwarding a packet and the control server for determining the routeinformation are separated from each other, the present invention ischaracterized by having a configuration to realize the redundancy of thecontrol channel due to the out-of-band control channel and the in-bandcontrol channel in the case where the switch receives from the controlserver the control message related to the flow entry registration andthe like based on the route information.

In FIG. 1, a network system where a data plane and a control plane eachconfiguring the control server for determining the switch and route toforward the packet are separated from each other is shown. Here, thecontrol channel for sending and receiving the control message connectsbetween the switch and the control server.

In FIG. 5, in order to determine the route information, the switch makesthe control channel redundant by not only being connected to the controlserver in the out-of-band control channel but constituting the in-bandcontrol channel in which the connect on with another control server canbe established via the network with use of the route for a normal datacommunication. Meanwhile, in the out-of band control channel, acommunication route dedicated to the control message is used.Additionally, in the in-band control channel, both of the communicationdata and the control message share the same communication route.

According to this manner, in the present invention, since sending andreceiving the control message in the out-of-band control channel and inthe in-band control channel, the switch can realize the redundancy ofthe control channel.

FIGS. 6A and 6B show a network system including: the switch forforwarding the packet; and a control server for managing and determiningthe route information.

In order to connect to the control server for mainly receiving thecontrol message in an initial configuration the switch is connected inthe out-of-band control channel separately from the network for dataforwarding, and the control server serves as the out-of-band controlserver for the switch.

The out-of-band control server requests a redundant configuration of thecontrol channel to another control server that is already known or to beconfigured, the control server existing on the network. The requestedanother control server serves as the in-band control server for theswitch.

With use of the redundancy control management unit, the out-of-bandcontrol server establishes an encrypted communication route between theout-of-band control server and the in-band control server, and notifiesthe in-band control server of the information on the switch managed byitself.

Moreover, the out-of-band control server notifies the switch managed byitself of the information on the in-band control server.

On the basis of the information on the in-band control server, theinformation being received from the out-of-band control server, theswitch requests the registration of the switch to the in-band controlserver. On the basis of the information on the switch preliminarilyreceived from the out-of-band control server, the in-band control servercertifies the switch and registers the switch. After the registration bythe in-band control server, the switch monitors the status of thein-band control server together with the out-of-band control server thatmainly receives the control message.

In the case where the reply is not issued from thecontinuously-monitored out-of-band control server, the switch determinesthat a trouble occurs in the out-of-band control server, switches fromthe out-of-band control server to the in-band control server, andreceives the control message from the in-band control server.

In this manner, the switch can receive the control message from thecontrol channel that is made redundant by the mainly-receivedout-of-band control server and another control server (the in-bandcontrol server).

<Supplement>

The features of the present invention are characterized in that: theredundant configuration by the out-of-band control channel and thein-band control channel; the establishment of connectability of thein-band control channel from the out-of-band control channel; theselection of the out-of-band control channel and the in-band controlchannel; and the switch between the out-of-band control channel and thein-band control channel.

In the present invention, the out-of-band control server configures andestablishes the control channel with the switch in the out-of-band, andsets the route information (the flow table) for the control channelbetween the switch and the in-band control server.

Then, the switch configures and establishes the control channel with thein-band control server, and constitutes the redundant configuration ofthe control server in the out-of-band control server and the in-bandcontrol server (using both of them). In addition, monitors the troublein the control channel, and switches in the trouble occurrence from anactive (a main series, an actual series) control channel to a stand-by(a waiting series, a reserve series) control channel.

The embodiment of the present invention has been described in detailabove; however, the present invention is not limited to theabove-described embodiment in actual, and modification within a range ofthe scope of the present invention is included in the present invention.

The present invention claims the priority based on Japanese ApplicationNo. 2010-000819, and the disclosure in Japanese Application No.2010-000819 is incorporated into the present invention by reference.

The invention claimed is:
 1. A network system comprising: a switchconfigured to forward a packet; an out-of-band control server configuredto send a control message to the switch via an out-of-band controlchannel to determine route information; and an in-band control serverconfigured to send a control message to the switch via an in-bandcontrol channel to determine route information, wherein the out-of-bandcontrol server is configured to: establish the out-of-band controlchannel with the switch; and set route information for the in-bandcontrol channel between the switch and the in-band control server. 2.The network system according to claim 1, wherein the switch isconfigured to: establish the in-band control channel with the in-bandcontrol server; and constitute a redundancy structure of a controlserver by using both of the out-of-band control server and the in-bandcontrol server.
 3. The network system according to claim 2, wherein theswitch is configured to: monitor a fault of both of the out-of-bandcontrol channel and the in-band control channel; and switch an activecontrol channel to a stand-by control channel among the out-of-bandcontrol channel and the in-band control channel when the fault occurs.4. A computer which is used as at least one of the switch, theout-of-band control server, and the in-band control server according tothe network system according to claim
 2. 5. The network system accordingto claim 2, wherein the switch is configured to: monitor a fault of bothof the out-of-band control channel and the in-band control channel; andswitch an active control channel to a stand-by control channel among theout-of-band control channel and the in-band control channel when thefault occurs.
 6. A computer which is used as at least one of the switch,the out-of-band control server, and the in-band control server accordingto the network system according to claim
 5. 7. A computer which is usedas at least one of the switch, the out-of-band control server, and thein-band control server according to the network system according toclaim
 2. 8. A network redundancy method comprising: forwarding a packetby a switch; sending a control message from an out-of-band controlserver to the switch via an out-of-band control channel to determineroute information; and sending a control message from an in-band controlserver to the switch via an in-band control channel to determine routeinformation; establishing the out-of-band control channel by theout-of-band control server with the switch; and setting routeinformation for the in-band control channel between the switch and thein-band control server by the out-of-bend control server.
 9. The networkredundancy method according to claim 8, further comprising: establishingthe in-band control channel with the in-band control server by theswitch; and constituting a redundancy structure of a control server bythe switch by using both of the out-of-band control server and thein-band control server.
 10. The network redundancy method according toclaim 9, further comprising: monitoring a fault of both of theout-of-band control channel and the in-band control channel by theswitch; and switching an active control channel to a stand-by controlchannel among the out-of-band control channel and the in-band controlchannel by the switch when the fault occurs.
 11. A non-transitoryrecording medium which stores a program when executed by a processor tomake a computer functions as each of the switch, the out-of-band controlserver, and the in-band control server according to the networkredundancy method according to claim
 9. 12. The network redundancymethod according to claim 8, further comprising: monitoring a fault ofboth of the out-of-band control channel and the in-band control channelby the switch; and switching an active control channel to a stand-bycontrol channel among the out-of-band control channel and the in-bandcontrol channel by the switch when the fault occurs.
 13. Anon-transitory recording medium which stores a program when executed bya processor to make a computer functions as each of the switch, theout-of-band control server, and the in-band control server according tothe network redundancy method according to claim 8.